SUGI 27: Use of the ROC Curve and the Bootstrap in Comparing Weighted Logistic Regression Models

In analyzing data from a survey, researchers often need to compare the effectiveness of several logistic regression models. The receiver operating characteristic curve offers one way to measure effectiveness of prediction, by calculating the area under the curve (AUC). We present a SAS macro for calculating AUC that takes the survey weights into account. For comparing logistic regression models, one needs to assess differences in AUC against the variation in the data. We demonstrate the use of the SAS SURVEYSELECT procedure to create a set of 1,000 bootstrap samples and give some background on the calculation of separate weights for each bootstrap sample. For each sample, the AUC macro is then used to calculate the AUC for each model. We show how to use the bootstrap results to assess the significance of the difference in predictive ability of the two models.